Field of Invention
Aspects of this invention are related to color imaging in surgical systems, and are more particularly related to demosaicing a frame of pixels.
Related Art
Many color cameras capture images using a single sensor array. The camera includes a color filter array and an image capture sensor. The color filter array filters the incoming light so that each pixel of the image capture sensor receives only one of the primary colors. Spatial resolution is sacrificed in view of the compact size of the single sensor array and the ability to generate full resolution images from the captured pixels.
A commonly used color filter array is referred to as a Bayer pattern color filter array. The Bayer pattern color filter array is described, for example, in U.S. Pat. No. 3,971,065, which is incorporated herein by reference.
FIG. 1A is a representation of part of a Bayer pattern color filter array. The Bayer pattern color filter array has a mosaic pattern. Each pixel in the Bayer pattern color filter array passes primarily a single primary color in a color space. In a color space with red, green, and blue color components, the pixels in the array selectively pass green light, red light, and blue light to the image capture sensor. Typically, the Bayer pattern color filter array is a mosaic with one-quarter of the total number of pixels being red (R) filters, one-half of the total number of pixels being green (G) filters; and one-quarter of the total number of pixels being blue (B) filters.
In the image acquired by the image capture sensor, no color component has the full resolution of the image capture sensor, i.e., the total number of pixels captured by the image capture sensor. As shown in FIG. 1B, the captured green pixels G are only one-half of the total number of pixels captured by the image capture sensor. In FIGS. 1C and 1D, the captured red pixels and blue pixels are each only one-quarter of the total number of pixels captured by the image capture sensor. To obtain a complete full resolution set of pixels for each of the color components, a process called demosaicing is commonly used. Demosaicing is a process used to reconstruct a full color image from the color samples that are output from an image capture sensor overlaid with a color filter array.
Since humans are sensitive to edge structures in an image, many adaptive demosaicing methods try to avoid interpolating across edges. Some examples of these methods are presented in U.S. Pat. No. 5,629,734 (disclosing “Adaptive Color Plane Interpolation in Single Sensor Color Electronic Camera”) and L. Zhang and X. Wu, “Color Demosaicing The Directional Linear Minimum Mean Square Error Estimation,” IEEE Trans. on Imaging Processing, 2005.
U.S. Pat. No. 5,629,734 describes an adaptive interpolation demosaicing process that utilizes a second order Laplacian filter. The filter uses the absolute value of a second order blue-red color gradient plus the absolute value of a first order green color difference to select an edge direction in the adaptive interpolation demosaicing process.
Each demosaicing process tries to improve on one or more of the factors that are important to demosaicing. A few factors considered in selecting a demosaicing process include: 1) reconstruction quality in terms of edges, 2) reconstruction quality in terms of color artifacts, 3) process complexity, and 4) reconstruction quality under noise. Unfortunately, many of the demosaicing processes that perform well with respect to the first, second, and fourth factors are complex, and it is challenging to implement them, without expensive hardware, in a surgical system that requires a real-time video display.